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RELIABILITY  ASSESSMENT  TEST  (RAT)  PROGRAM 
LIFE  DATA  ANALYSIS  METHODS 


PURPOSE 

The  purpose  of  this  report  is  to  discuss  two  life  data  analysis  methods  which  will  be  employed  in  the 
Global  Positioning  System  (GPS)  Block  HR  Rubidium  Atomic  Frequency  Standard  (RAFS),  Reliability 
Assessment  Test  program.  The  intent  here  is  not  to  perform  analysis  of  the  currently  available  RAFS 
life  data  but  to  describe  recommendations  for  the  analysis  methodology.  Methods  for  parameter 
modeling  or  analysis  of  the  RAFS  performance  (accuracy,  stability,  or  drift)  will  not  be  considered. 

The  procedures  proposed  suggest  a  method  for  inferring  the  expected  useful  life  of  the  Block  IIR 
RAFS.  It  is  expected  that  there  will  be  enhancements  to,  or  alternative  analysis  methods  proposed  as 
Reliability  Assessment  Test  (RAT)  RAFS  lifetime  data  are  accumulated.  In  addition,  the  analysis 
results  could  be  used  to  support  RAFS  risk  analysis.  Risk  analyses  are  the  calculations  that  predict 
present  and  future  risk.  Present  risk  is  the  expected  number  of  failures  from  the  current  population  of 
RAFS,  while  future  risk  is  a  prediction  of  the  number  of  RAFS  failures  over  some  future  period  of 
time.  Predictions  could  be  made  for  common  or  multiple  failure  modes.  This  will  be  possible  if  the 
appropriate  life  distribution  and  its  parameters  can  be  established  and  that  the  Block  IIR  RAFS  becomes 
the  primary  space  borne  atomic  reference  in  the  GPS  satellite  constellation. 

RAT  LIFE  DATA  ANALYSIS 

An  objective  of  the  RAFS  RAT  is  to  perform  lifetime  data  analysis  of  the  Block  IIR  RAFS.  The 
results  are  to  be  used  to  develop  estimates  of  the  RAFS  mean  life  based  on  actual  life  time 
observations.  Lifetime  is  defined  as  the  period  from  the  beginning  of  on-orbit  operations  to  the  time 
where  a  particular  failure  mode  has  caused  the  clock  to  be  considered  unfit  for  use.  Unfitness  for  use 
does  not  require  a  catastrophic  failure.  Degradation  in  performance  below  an  acceptable  level  will  be 
identified  as  a  failure.  The  RAT’s  period  of  performance  is  only  three  years.  Therefore,  it  is  expected 
that  there  will  be  no,  or  at  best,  very  few  failures  observed.  In  the  case  where  there  are  no  failures  it 
will  be  impossible  to  estimate  the  life  distribution  parameters.  In  the  case  of  few  failures,  parameter 
estimates  would  be  suspect  and  subject  to  considerable  uncertainty.  The  precision  or  accuracy  of  the 
estimates  are  directly  related  to  the  number  of  life  times  (actual  failures)  observed.  It  is  anticipated  that 
actual  failure  data  will  be  limited.  Therefore,  the  approaches  are  based  on  a  no  or  few  failure  situation. 
The  approaches  are  designed  to  establish  lower  bound  confidence  levels,  rather  than  point  estimates  of 
the  RAFS’s  mean-life  (0)  or  characteristic  life  (r|).  The  lower  bound  confidence  level  is  the  probability 
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that  a  given  parameter  lies  above  a  lower  bound.  One  point  should  be  made  regarding  the  precision  of 
the  lower  bounds  and  a  specified  confidence  level;  they  are  inversely  related.  Figures  (1)  to  (5) 
illustrate  this.  The  RAFS  life  data  analysis  methods  are  based  on  procedures  where  the  times-to- 
failures  are  Exponential  (one  parameter)  and  Weibull  (two  parameter)  distributed.  The  procedures 
assume  that  all  RAFS  operating  times  (both  the  ground  test  and  in  orbit)  will  be  considered  part  of  the 
analysis  program. 


LIFE  TIME  ANALYSIS  PROBABILITY  DISTRIBUTIONS 

Exponential: 

The  Exponential  distribution  is  the  most  commonly  used  distribution  in  time-to-failure  analysis  of 
electronic  systems.  Some  of  the  reasons  for  its  popularity  are: 

1.  Simplicity;  data  analysis  methods  are  simpler  than  many  other  distributions. 

2.  It  has  been  sanctified  by  years  of  research,  usage,  and  technical  publications. 

3.  Most  government  contracts  specify  the  use  of  Military  Handbook  217  which  is  based  on  the 
Exponential  distribution. 

Weibull: 

The  Weibull  distribution  is  also  a  widely  used  distribution  for  life  testing  applications.  The  Weibull 
distribution  describes  the  life  characteristics  of  non-repairable  parts  and  components.  Depending  on  its 
shape  (P)  and  scale  (r|)  parameters,  it  can  fit  many  sets  of  lifetime  data.  Some  contemporary  reliability 
and  life  data  analysis  authorities  advocate  the  use  of  the  Weibull  over  the  Exponential.  The  use  of  the 
Weibull  is  gaining  in  popularity  partly  because  of  computer  statistical  analysis  software. 

WEIBULL  DISTRIBUTION  CHARACTERISTICS 

Most  people  in  the  defense  industry  are  familiar  with  Exponential  life  data  applications.  However, 
many  are  unfamiliar  with  use  of  the  Weibull  distribution  or  its  applications  in  life  data  analysis.  The 
Block  HR  prime  contractor  (LMMC)  has  requested  that  Weibull  analysis  be  considered  as  an  adjunct  to 
any  analysis  based  on  Exponential  distribution. 

There  are  two  approaches,  probability  plotting  (graphical)  and  statistical,  for  performing  Weibull 
life  data  analysis.  Probability  plotting  is  the  technique  predominately  used  because  of  its  simplicity  and 
applicability  to  a  situation  where  only  scant  data  is  available.  Additionally,  software  is  readily  available 
for  doing  probability  plotting  of  Weibull  distributed  data.  Figure  1  is  a  Weibull  probability  plot  using 
Weibull  analysis  software.  The  abscissa  is  the  order  times  to  failure  and  the  ordinate  is  -ln[l-F(t)], 
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where  F(t)  is  the  probability  of  occurrence.  The  data  analyzed  was  derived  from  data  known  to  be 
Weibull  distributed. 

Statistical  analysis  technique  followed  development  of  the  graphical  technique.  The  use  of 
statistical  procedures  to  estimate  distribution  parameters  adds  precision  to  the  analysis  and  removes  the 
subjectivity  which  occurs  when  reading  probability  plots.  There  are  some  drawbacks  with  the 
statistical  analysis  approach.  It  requires  a  much  broader  appreciation  of  mathematical  statistics  and 
probably  won’t  provide  much  insight  into  what  the  RAT  is  trying  to  accomplish.  As  a  result,  statistical 
analysis  techniques  will  not  be  initially  considered,  but  may  be  used  as  failure  data  is  accumulated. 

As  previously  stated,  the  Weibull  distribution  is  extremely  flexible  and  analysis  can  be  performed 
with  small  data  sets.  Inference  can  be  made  from  an  analysis  of  a  few  data  points.  The  following 
information  can  be  obtained  directly  from  a  probability  plot: 

1.  The  estimates  of  the  shape  parameter  P  or  sometimes  called  the  slope. 

2.  The  estimates  of  the  population  characteristic  life  (t|)  of  which  63.2%  of  the  population  will 
have  a  life  less  than  or  equal  to. 

3.  The  percentage  of  the  population  that  is  expected  to  fail  by  some  time  (t). 

4.  The  Weibull  mean  (or  Mean-Time-To-Failure  (MTTF))  can  be  estimated  based  on  the 

following  functional  relationship  between  the  Weibull  mean,  shape,  and  scale  parameters, 
MTTF=ri[r(l  +  l/p)]. 

In  figure  1  the  characteristic  life  can  be  read  directly  from  the  abscissa,  at  the  point,  were  -ln[l-F(t)] 
equals  one  (I)  on  the  ordinate  intersects  with  the  fitted  line.  With  a  little  bit  of  algebra  the  probability 
of  occurrence,  F(t)  can  be  computed  as  1  -e  _1  or  .632.  To  estimate  what  percentage  (x)  of  a 
population  is  predicted  to  have  a  life  less  than  or  equal  to  some  time  (t)  take  the  value  (y)  on  the 
ordinate,  which  is  equal  to  -ln[l-x].  Then  read  the  life  time  on  the  abscissa  where  (y)  intersects  the 
fitted  line.  Of  course,  the  reverse  of  this  computation  can  be  performed  based  on  some  specified  life 
time  and  solving  for  F(t).  Beta  can  be  determined  by  taking  the  slope  of  the  fitted  line.  However,  the 
software  will  do  this  and  provide  estimates  of  (r|)  and  (P)  in  the  analysis  summary. 


THE  EXPONENTIALLY  DISTRIBUTED  RAFS  LIFE  DATA  ANALYSIS  METHODOLOGY 

The  point  estimates  of  the  RAFS’s  MTTF  will  not  be  established  but  rather,  instead,  a  lower 
confidence  bound  on  the  MTTF  will  be.  This  approach  is  based  on  the  work  by  two  statistical 
researchers  Benjamin  Epstein  and  Milton  Sobel  and  is  described  in  their  landmark  paper  “Life 
Testing.”  Epstein  and  Sobel  showed  that  if  the  time-to-failure  (t)  is  exponentially  distributed  the  random 

2r0 

variable  —  is  chi-squared  distributed.  0  csl  is  the  MTTF  based  on  a  sample,  while  0  lrue  is  the 
MTTF  of  tie  population.  a  is  the  level  of  significance.  With  the  probability  statement  being 
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and  using  some  algebra  the  lower  one-sided  confidence  bound  for  a  time  censor  life  test  is 


IT 


Xa,2r+2 


*0 


=  l  -a .  r  equals  the  number  of  observed  failures  and  setting  0  equals  to 


the  mission  required  MTTF  in  years.  The  required  total  accumulated  operating  time  (Ta)  at  the 
100%(l-a  )  lower  confidence  bound  can  be  computed  as  MTTF*. 5  2(r+i)-  The  degrees  of 

freedom  for  a  time  censor  test  (test  terminated  at  a  specific  time)  is  2(r+ 1).  The  r  + 1  rather  than  r 
degrees  of  freedom  are  based  on  the  imminent  failure  assumption.  For  example,  with  a  specified 
MTTF  of  7.5  years  and  assuming  zero  failures,  the  total  test  time  required  for  a  80%  lower  confidence 
bound  would  be  7.5*.5jc222  or  12.07078  years  total  accumulated  operating  time.  Additionally,  using 
the  equation  for  computing  Ta  and  the  specified  MTTF,  the  level  of  confidence  can  be  determined  on 
the  lower  bound  given  the  life  data.  For  example,  at  the  end  of  three  years  we  have  the  following: 
three  units  with  three  years;  one  unit  has  two  years  and  one  unit  has  one  year  for  a  total  of  12  years 
(3*3+2+ 1).  Then  0.2.  3.2  on  the  chi-squared  tables  at  2  degrees  of  freedom  is  an  a  of  .2 

or  a  1 -a  of  .8,  which  is  approximately  80%  confidence  lower  bound  on  the  MTTF  of  7.5  years. 

Figure  2  provides  a  plot  of  the  confidence  level  verses  MTTF  estimates.  Notice  the  inverse 
relationship.  The  estimates  of  the  MTTF  decreases  as  the  confidence  level  increases.  Figure  3 
provides  plots  of  the  total  accumulated  operating  time  required  to  establish  the  one-sided  lower  bound 
%  confidence  for  different  selected  MTTFs.  Note  the  direct  relationship.  For  a  given  MTTF,  as  the 
confidence  level  increases  the  required  total  accumulated  operating  time  increases.  One  of  the  benefits 
of  this  approach  is  its  simplicity  and  straightforwardness.  Computations  are  only  a  function  of  total 
accumulate  operating  time. 


There  are  some  drawbacks  associated  with  the  use  of  the  exponential  distribution.  One  drawback  is 
that  the  inference  procedures  lack  robustness.  The  decision  to  propose  a  Exponentially  distributed 
time-to-failure  analysis  method  was  based  on  conclusions  resulting  from  various  technical  journals. 

The  journals  revealed  that  while  there  is  some  disagreement  as  to  whether  or  not  the  assumption  of 
exponentiality  of  time-to-failure  for  electronics  component  may  be  questionable,  all  authors  basically 
agreed  that  time-to-failure  for  systems  tend  to  be  exponentially  distributed. 


WEIBULL  DISTRIBUTED  RAFS  LIFE-DATA  ANALYSIS  METHODOLOGY 


The  proposed  analysis  procedures,  based  on  the  Weibull  distribution,  were  developed  by  Z.  Huang 
and  A.  Porter  of  the  Rockwell  International,  Rocketdyne  Division.  The  analysis  method  was  published 
in  the  paper  “Lower  Bound  On  Reliability  for  Weibull  Distribution  When  Shape  Parameter  Is  Not 
Estimated  Accurately”.  The  method  provides  techniques  for  performing  life-data  analysis  with  no  or 
limited  life-time  (failure)  data.  The  consequences  of  life-time  analysis  under  these  situations  were 
already  commented  on.  Traditionally,  under  such  conditions  the  analyst  assumes  P  is  known  and  q  and 
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other  quantities  are  estimated  from  the  data.  This  approach  would  lead  to  a  lot  of  questions  and  would 
be  difficult  to  justify.  An  alternative  method  developed  by  Huang  and  Porter  attempts  to  overcome  the 
drawbacks  of  the  “Assumed  (P)  Method”.  The  approach  provides  methods  where  under  general 
conditions,  reliability  and  allowable  life  limits  (tolerance  limit)  will  have  unique  global  minima  over  a 
range  of  p.  The  use  of  the  allowable  life  limit  lower  bound  method  is  proposed  for  RAFS  life  time  data 
analysis.  The  method  demonstrates  that  for  a  given  reliability  (assumed  to  be  known  and  constant)  the 
life  limit  experiences  a  minimum  value  as  a  function  of  p.  The  life  limit  reaches  a  minimum  at  some 
value  of  p.  The  method  is  based  on  the  following  statistical  fact.  If  the  time  to  failure  (t)  is  Weibull 
distributed  then  tp  is  exponential  distributed  with  the  MTTF  parameter  0=  r|p .  To  calculate  a  chi- 
squared  lower  confidence  limit  use 


=  l  -a  and  an  estimate  of  the  characteristic  life 


T)  IS 


E'f 

i=i 


•5X« 


.  tj  is  the  i-th  life  time,  1-a  is  the  confidence  level,  and  n  is  the 


,2r+2  ) 


number  of  observations.  Put  this  result  into  the  Weibull  estimated  reliability  (R),  equation 
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*(p) 


Limit  Lower  Bound. 


v  E<'  , 
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and  solve  for  T 
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1=1  1  .  The  result  is  the  Allowable  Life 
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The  use  of  the  Huang/Rorter  Allowable  Life  Limit  Lower  Bound  method  is  conditioned  on  the 

y 

following  inequality  k  <  -  —  <  n.  k  is  the  number  of  observations  which  are  equal  to  the 

maximum  of  all  of  the  observations  (i.e.  t,  =  t2  =t  3  = . tn  =tmax).  This  condition  will  be  violated  at 

least  initially  because  all  the  operating  times  will  be  approximately  the  same  at  least  for  the  first  three 
units  (ground  test  and  first  flight  units).  Based  on  the  method’s  condition,  figure  4  shows  how  the 
confidence  level,  number  of  observations,  and  the  value  of  (k)  are  directly  related. 


Figure  5  provides  plots  illustrating  the  results  obtained  from  analysis  using  the  Huang/Porter 
method  for  the  Allowable  Life  Limit  Lower  Bounds  for  a  constant  reliability  of  80%.  Five  different 
confidence  levels  (50%  to  90%)  are  provided  to  illustrate  the  impact  different  confidence  levels  will 
have  on  the  analysis.  The  lower  bound  on  the  allowable  life  limit  is  inversely  related  to  the  confidence 
level.  The  plots  show  that  for  a  given  reliability  and  confidence  level,  the  device  could  be  operated  for 
at  least  the  minimum  value  (T)  without  failure.  The  worst-case  values  where  p  equals  P0  are  provided 
in  the  chart  table.  The  data  used  for  the  plots  was  from  an  expanded  Huang  and  Porter  data  set. 


A  concern  with  this  method  was  that  it  assumes  all  test  units  will  be  subject  to  a  common  failure 
mode.  In  reality,  the  RAFS  will  probably  exhibit  competing  failure  modes.  Huang  in  his  paper 
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“Conditional  Reliability  Lower  Bound  for  Weibull  Distribution  Without  Estimating  Shape  Parameter” 
showed  that  the  method  is  applicable  to  competing  failure  modes  with  the  same  shape  parameter  (P)  but 
different  scale  parameters  (r|;). 

Competing  failure  modes  is  a  situation  where  a  unit  can  fail  from  m  different  independent  failure 
modes.  For  example,  suppose  that  a  unit  exhibits  k  modes  of  failure  (m,,m2,  ...mk)  and  that  the 
following  random  unit  lifetime  occurs.  When  the  unit  begins  operation,  each  failure  mode 
simultaneously  generates  a  random  lifetime  that  is  independent  of  the  other  failure  modes.  Thus,  in 
effect,  k  lifetimes  have  simultaneously  begun.  Failure  of  the  unit  occurs  as  soon  as  any  one  of  the 
lifetimes  is  realized.  In  effect,  the  life  length  of  the  unit  is  equal  to  the  minimum  of  the  k  random 
lifetimes. 

CONCLUSION 

Eventually,  a  more  traditional  analysis  methodology,  based  on  the  Weibull  distributed  time  to 
failure,  is  expected  to  be  the  preferred  method  for  the  RAT.  However,  it  is  expected  that  all  analysis 
will  include  lower  confidence  limit  estimations  of  distribution  parameters  rather  than  point  estimates. 

In  the  mean  time,  beginning  at  the  completion  of  the  first  year  of  the  RAT,  the  application  of  the 
methodology,  based  on  the  research  of  Epstein  &  Sobel,  will  be  used  and  updated  on  six  month 
intervals.  The  time  to  failure  data  is  assumed  to  be  exponentially  distributed.  The  Huang  /Porter 
Weibull  distributed  Method  will  not  be  used  initially  to  support  the  RAT  life  analysis.  It  will  be  used 
when  the  sample  population  size  increases  to  a  point  where  the  conditional  requirements  for  the  use  of 
the  Allowable  Life-limit  Lower  Bound  method  are  met.  Research  will  continue  for  alternate  analysis 
approaches.  This  on-going  research  will  be  done  via  searches  through  technical  literature  and  state-of- 
the-art  data  analysis  software  packages. 
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Weibull  Probability  Plot  for  Exponentially  Distributed  Time-To-Failure 
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Total  Accumulated  Time  in  Years 


Accumulated  Time  Vs  %  Confidence  for 
a  Specified  MTTF  with  Zero  Failures 


Figure  #3 


MTTF=  7.5  Yrs  -q-  MTTF=6  Yrs 


■X-  MTTF=5  Yrs 


^  MTTF=  4  Yrs 


%  Confidence  Vs  Nbr  of  Observations 
Figure  #4 
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(k)  Less  Than 


Allowable  Life  Limit  Lower  Bound  (T) 


50%  to  90%  Lower  Confidence  Bound 
Figure  #5 


50%  T=38.919  Beta=  19.377  -  60%  T=38.218  Beta=12.059  -  70%  T=37.1673  Beta=8.025 


- 80%  T=35.535  Beta=5.561  -  90%  T=32.44  Beta=2.86 
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Danzy  and  Smathers 
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